Section: New Results

Activity Description Language

Participants : Daniel Gaffé, Sabine Moisan, Annie Ressouche, Jean-Paul Rigault, Ines Sarray.

Activity Recognition aims at recognizing and understanding sequences of actions and movements of mobile objects (human beings, animals or artefacts), that follow the predefined model of an activity. We propose to describe activities as a series of actions, triggered and driven by environmental events.

Due to the large range of application domains (surveillance, safety, health care ...), we propose a generic approach to design activity recognition systems that interact continously with their environment and react to its stimuli at run-time. Such recognition system must satisfy stringent requirements: dependability, real time, cost effectiveness, security and safety, correctness, completeness ... To enforce most of these properties, our approach is to base the configuration of the system as well as its execution on formal techniques. We chose the Synchronous Approach which provides formal bases to perform static analysis, verification and validation, but also direct implementation.

Based on the synchronous approach, we have created a new user-oriented activity description language (named ADeL) to express activities and to automatically generate recognition automata. This language relies on two formal semantics, a behavioral and an equational one. This year, we continued to work on this topic: we improved both the syntax of the ADeL language to be easier to use by non computer-scientists and its semantics to generate synchronous automata.

As the world is not synchronous and since we are working with the synchronous paradigm, we have to face the classical problem of sampling. Our systems have to deal with asynchronous events coming from the environment. This year we started to define an asynchronous/synchronous transformer component, that we call Synchronizer, to transform asynchronous sensor events into synchronous “instants”.

Activity Description Language (ADeL)

ADeL is a (synchronous) language that allows non-computer scientists to describe activities and behaviors to be recognized. It is a modular and hierarchical language, which means that activities can be simple or composed of one or more sub-activities. ADeL has the notions of (typed) roles, events and sub-activities, flow of control... It supports parallelism, variants (choices), and repetitions. ADeL relies on a set of formally specified control and temporal operators.

We provide our language with both a graphical and textual format. We propose a graphical tool which displays several windows, mainly to declare the scene where the activity will take place (zones, roles and equipment) and to describe the activity (expected events and “story board”). However, it may be difficult to express complex activities in a purely graphical way, thus we also provide an equivalent textual form.

This year we conducted a preliminary heuristic evaluation to define the layout of the graphical tool in collaboration with our ergonomist partners from LudoTIC. An example of a scene description window for a medical application (serious game) is shown in Fig. 32. We made a first positive user test with a doctor, more extensive user tests are planned.

Synchronizer

The main drawback of the synchronous paradigm is that the world is not synchronous in general. Thus it requires to transform asynchronous physical flows of events into a succession of discrete instants.

This year, we specified a Synchronizer component to address this issue. The Synchronizer filters physical asynchronous events, decides which ones may be considered as “simultaneous” and aggregates the latter into logical instants. The sequence of these instants constitutes the logical time of our recognition systems. In general, no exact simultaneity decision algorithm exists but several empirical strategies and heuristics may be used for determining instant boundaries. We have to take into account these parameters in the Synchronizer specification. This year we completed the UML specification of the Synchronizer and we started its first implementation.

Semantics

We defined two semantics for the ADeL language [34]. First, conditional rewriting rules are a classical and rather natural way to formally express the intuitive semantics. This form of behavioral semantics gives an abstract description and a clear interpretation of a program behavior. However it is not convenient as an implementation basis nor suitable for proofs (e.g., model-checking). Hence we also define an equational semantics which maps an ADeL program into a Boolean equation system representing its finite state machine. The ADeL compiler can easily translate this equation system into an efficient code not only for our runtime recognition component, but also for other tools such as model-checkers.

Since we have two different semantics, it is mandatory to establish their relationship. In fact we proved that the execution of a program based on the equational semantics also conforms to the behavioral semantics [34], [38].